1. Field of the Invention
The invention relates to a method for production of reduced-error, high-resolution and improved-contrast images from image sequences with low-resolution images, which are obtained by an image sensor with variable recording parameters.
2. Discussion of Background Information
Imaging sensor systems such as camera systems, irrespective of whether they are in the visible range or in the IR range, are subject to limits both with respect to the final image resolution (number of pixels) and the final dynamic range (contrast range). These quality factors are defined primarily by technical and financial constraints. Further constraints result from the type of system in which the imaging sensor system is implemented. For example, in flying platforms, the image resolution and the dynamic range of the sensor system cannot be increased indefinitely since this would significantly increase both the amount of data produced and the requirements for any downstream data transmission devices.
Contrary to general opinion, an increase in the number of the pixels in an image on its own does not automatically also mean an increase in the information content of the image, and in fact this is dependent on the capability to identify detail in the image.
A number of techniques are known whose aim is to improve the resolution and/or the dynamic range of images produced by imaging sensor systems with a given number of pixels, both of individual images and of image sequences, for example in the case of moving images.
In addition to the traditional techniques of image processing of individual images such as brightness normalization, contrast improvement, focusing, and noise suppression, methods are known in which a plurality of images in an image sequence (image stack) are subjected to common processing, so-called “fusion”, in order to obtain one or more improved-quality images or an improved-quality image sequence from low-resolution images in an image sequence.
The aim of a first known type of quality improvement is to improve the dynamic range by using complementary information which is contained in a plurality of images in an image sequence, which images have been recorded with one sensor parameter being varied, for example the exposure, in order to obtain an image with an improved dynamic range. This type of dynamic processing is known as high-dynamic-range reconstruction processing (HDR) and is described, for example, in E. Reinhard, G. Ward, S. Pattanaik, P. Debevec, High Dynamic Range Imaging: Acquisition, Display and Image-based Lighting, Morgan Kaufmann, 2006, ISBN: 0125852630.
A further known type of image processing has the aim of processing redundant information contained in a plurality of low-resolution images contained in an image sequence or an image stack, so as to achieve high resolution and/or a reduction in errors. In this case, the recording parameters for example the exposure etc., are typically maintained for all the images in the image sequence that is used. This type of image quality improvement is known as superresolution processing, and is described in detail, for example, in M. Gevrekci, B. K. Gunturk, Super-Resolution Approaches For Photometrically Diverse Image Sequences, ICASSP, 2007, and in T. Pham, Spatiotonal Adaptivity in Super-Resolution of Under-Sampled Image Sequences, PhD thesis, Delft University of Technology, 2006 or in S. Farsiu, D. Robinson, M. Elad, P. Milanfar, Advances and Challenges in Super-Resolution, IJIST, 2004.
In known fusion methods for increasing the image quality, which are based on processing of an image sequence or of an image stack, use is generally also made of registration, that is to say mutual alignment of the individual images with respect to one another, typically with respect to a reference image. Registration methods such as these are described, for example, in B. Zitova, J. Flusser, Image Registration Methods: A Survey, 2003.
The disadvantage of the use of these methods has until now been that they are in each case applied only to a limited portion of the image information which can be obtained by the imaging sensor system. For example, the traditional image processing methods described initially operate only on the basis of individual images. Until now, the fusion methods for high-dynamic-range reconstruction processing (HDR) and for superresolution processing (SR) have been carried out only in the sense either of processing of the redundant information, in particular in superresoltuion processing, or exclusively the complementary information with the variation of sensor parameters, that is to say in particular high-dynamic-range reconstruction processing (HDR).